-
August 26th, 2026, 03:15 AM
#1
Senior Hostboard Member
Managing a distributed job queue with PostgreSQL
I'm building a job queue system backed by PostgreSQL and running into the usual challenges around concurrent workers. Right now I'm using SELECT ... FOR UPDATE SKIP LOCKED to prevent multiple workers from grabbing the same job, which works well enough at low volume. As worker count scales up though, I'm seeing lock contention and occasional delays in job pickup. A few things I'm uncertain about: whether partitioning the jobs table by status actually helps at this scale, how others handle stuck or zombie jobs without aggressive polling, and whether advisory locks offer any real advantage over row-level locks in this context. Would love to hear from anyone who's run this in production — particularly around dead letter queues, retry logic, and whether you eventually migrated away from PostgreSQL entirely for this use case.
Tags for this Thread
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
Bookmarks